Dense layer

A hidden layer in which each neuron is connected to every neuron in the subsequent hidden layer.¹

Footnotes

Precise Zero-Shot Dense Retrieval without Relevance Labels “While dense retrieval has been shown effective and efficient across tasks and languages, it remains difficult t…”
Connectionist approaches to language “Another processing principle concerns the transformations of activity patterns from one layer of units to the n…”
Memory Layers at Scale “On downstream tasks, language models augmented with our improved memory layer outperform dense models with more…”
합성곱 신경망 “A neural network in which at least one layer is a convolutional layer.”
Embedding layer “A special hidden layer that trains on a high-dimensional categorical feature to gradually learn a lower dimensi…”
역전파 “During the backward pass (backpropagation), the system reduces loss by adjusting the weights of all the neurons…”
계산주의-연결주의 논쟁 관련 절: 각종 메모
Sparse features “If the one-hot encoding is big, you might put an embedding layer on top of the one-hot encoding for greater eff…”
Scaling Monosemanticity: Extracting Interpretable Features from Claude 3 Sonnet “Eight months ago, we demonstrated that sparse autoencoders could recover monosemantic features from a small one…”